feat(cli): dispatch jobs asynchronously and push results/logs to the caller - #1835
Draft
robert-ursu wants to merge 1 commit into
Draft
feat(cli): dispatch jobs asynchronously and push results/logs to the caller#1835robert-ursu wants to merge 1 commit into
robert-ursu wants to merge 1 commit into
Conversation
…caller
`uipath server` used to block for a whole job and leave its outcome on disk for
the caller to find. StartJob now enqueues the work and returns; the server
pushes logs while the job runs and the terminal result when it finishes.
Behaviour is a pure function of the request: a caller that supplies
`resultCallbackSocket` gets async dispatch, one that does not gets the original
blocking call, byte for byte. No handshake, no capability gate on the dispatch
path -- an older caller cannot send the field and could not serve the callback
if it did.
POST {callback}/api/python/jobs/{jobKey}/result
POST {callback}/api/python/jobs/{jobKey}/logs
Execution stays serialised behind the process-wide lock: a job mutates process
globals (logging handlers, OTel providers, env, cwd), so queueing changes who
waits, not how many run.
Real cancellation. run/debug/eval each drive their own event loop via
asyncio.run inside the to_thread worker, so a job IS an event loop. They now go
through `run_job_loop`, which publishes that loop to a JobControl carried on a
ContextVar (to_thread propagates contextvars, so no monkeypatching). StopJob
cancels the root task, which unwinds the runtime cooperatively -- its context
managers still run, so output.json is still written and the file fallback still
works. Escalates to a full loop sweep, then reports False rather than claiming
a stop that did not happen.
Also fixes two pre-existing defects that made the API result meaningless:
_run_command_isolated hardcoded ExitCode 0 while click RETURNS ctx.exit(N)'s
code under standalone_mode=False, so every ConsoleLogger.error path reported
success; and the HTTP body now carries exitCode, which is the field the
un-upgraded .NET handler already reads.
Notes for review:
- Server diagnostics go to a stderr handle bound at import, NOT ConsoleLogger.
ConsoleLogger resolves sys.stdout at call time, and the runtime's interceptor
has replaced it with a writer feeding the job's execution.log -- which the
tailer then reads and posts back. With the callback down that is a
self-feeding loop.
- A 4xx from the callback is REJECTED, not UNREACHABLE: the caller is up and has
moved on, so retrying cannot help and must not trip the shutdown path.
- The log tailer holds back an unterminated tail; a handler writes the record
and only then flushes, so a poll can otherwise split one line into two
entries that cannot be rejoined.
- StopJob takes resume_version as a trailing optional parameter rather than a
DTO: uipath-ipc ignores a surplus wire arg and defaults a missing one, so old
and new peers interoperate in both directions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| connector=conn, timeout=ClientTimeout(total=CALLBACK_TIMEOUT_SECONDS) | ||
| ) as session: | ||
| async with session.post( | ||
| f"http://localhost{path}", json=payload |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Summary
uipath serverused to block for a whole job and leave its outcome on disk for the caller to find.StartJobnow enqueues and returns; the server pushes logs while the job runs and the terminal result when it finishes.Behaviour is a pure function of the request:
resultCallbackSocketpresent202 {"disposition":"accepted"}, result pushed laterThat is the whole compatibility story — an older caller cannot send the field, and could not serve the callback if it did. No handshake on the dispatch path.
Execution stays serialised behind the process-wide lock: a job mutates process globals (logging handlers, OTel providers, env, cwd), so queueing changes who waits, not how many run.
Real cancellation
run/debug/evaleach drive their own event loop viaasyncio.runinside theto_threadworker — so a job is an event loop. They now go throughrun_job_loop, which publishes that loop to aJobControlcarried on aContextVar(to_threadpropagates contextvars, so no monkeypatching, and it isasyncio.runverbatim when no control is in scope).StopJobcancels the root task only, which unwinds the runtime cooperatively — its context managers still run, sooutput.jsonis still written and the file fallback still works. Cancelling every task would abort that cleanup mid-write. It escalates to a full sweep, then reportsFalserather than claiming a stop that did not happen.A cancelled job is reported Stopped, not Faulted: the runtime records it as
FAULTED/ERROR_CancelledError, which is the wrong story for a stop the caller asked for.Also fixes two pre-existing defects
Both made the API result meaningless, and were masked only because the caller overwrote the status from the file afterwards:
_run_command_isolatedhardcodedExitCode: 0, while click returnsctx.exit(N)'s code understandalone_mode=False— so everyConsoleLogger.errorpath reported success.exitCode, which is the field the un-upgraded .NET handler already reads. That alone makes an old handler correct against a new runtime with no .NET change.Worth a careful look
ConsoleLogger.ConsoleLoggerresolvessys.stdoutat call time, and the runtime's interceptor has replaced it with a writer feeding the job'sexecution.log— which the tailer reads and posts back. With the callback down that is a self-feeding loop.REJECTED, notUNREACHABLE. The caller is up and has moved on; retrying cannot help and must not trip the shutdown path.StopJobtakesresume_versionas a trailing optional parameter, not a DTO. uipath-ipc ignores a surplus wire arg and defaults a missing one, so old and new peers interoperate both ways; a DTO would break an old venv loudly and a new one silently.UiPathRuntimeLogsInterceptor._clean_all_handlersstrips every handler but its own, anduipath-runtimeships on its own release train. The file is still written unchanged.Testing
uv run pytest tests/cligreen (~1400 tests; ~60 new acrosstest_server_{result,async,logs,callbacks,cancellation}.py).ruff check,ruff format --check,mypyclean.The cancellation tests use commands shaped like the real ones and assert the job's
finallyactually ran — an early cut usedasyncio.run(loop_factory=...), which is 3.12+, and every one of them failed on the 3.11 venv. Now onasyncio.Runner.Known gaps (deliberate)
state.dbstays on disk — agent code opens that path directly.🤖 Generated with Claude Code